[REVIEW] Define and implement new stream compaction APIs `copy_if`, `drop_nulls`, `apply_boolean_mask`, `drop_duplicate` and `unique_count`. #3303

rgsl888prabhu · 2019-11-05T21:05:57Z

Define and implement new stream compaction APIs copy_if, drop_nulls, apply_boolean_mask, drop_duplicate and unique_count.

NOTE: All the APIs supports string

close #2948
close #3415

…948_adding_copy_if

codecov · 2019-11-05T22:33:01Z

Codecov Report

Merging #3303 into branch-0.11 will not change coverage.
The diff coverage is n/a.

@@             Coverage Diff              @@
##           branch-0.11    #3303   +/-   ##
============================================
  Coverage        87.36%   87.36%           
============================================
  Files               49       49           
  Lines             9295     9295           
============================================
  Hits              8121     8121           
  Misses            1174     1174

Impacted Files	Coverage Δ
python/cudf/cudf/core/column/column.py	`87.06% <0%> (ø)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 55596e3...5e64ad3. Read the comment docs.

harrism

Looks good, except for the mystery of the missing apply_boolean_mask!

cpp/include/cudf/detail/copy_if.cuh

cpp/include/cudf/table/table_view.hpp

cpp/include/cudf/wrappers/bool.hpp

cpp/include/cudf/wrappers/timestamps.hpp

cpp/src/stream_compaction/drop_nulls.cu

rgsl888prabhu · 2019-11-06T04:28:59Z

Looks good, except for the mystery of the missing apply_boolean_mask!

@harrism Ouch, I thought it had different kernel and was planning a PR along with drop_duplicate. I will update this PR with that one as well.

harrism · 2019-11-06T04:32:14Z

SHouldn't be too hard -- it's a wrapper around copy_if like drop_nulls is.

cpp/include/cudf/detail/gather.cuh

cpp/include/cudf/detail/sorting.hpp

harrism · 2019-11-18T22:17:50Z

cpp/include/cudf/detail/stream_compaction.hpp

+ */
+std::unique_ptr<experimental::table>
+    drop_nulls(table_view const& input,
+               table_view const& keys,


I'm confused. Jake asked you to changed this to a vector of indices, and you agreed, but the change hasn't been made and the conversation is resolved. Unresolving to confirm.

cpp/include/cudf/detail/utilities/cuda.cuh

cpp/src/stream_compaction/drop_duplicates.cu

cpp/include/cudf/detail/copy_if.cuh

cpp/include/cudf/detail/gather.cuh

cpp/include/cudf/table/table_device_view.cuh

cpp/src/stream_compaction/apply_boolean_mask.cu

harrism · 2019-11-20T00:45:11Z

rerun tests

cpp/tests/stream_compaction/drop_duplicates_tests.cu

harrism

Fantastic work, Ram!

rgsl888prabhu · 2019-11-21T15:40:55Z

@jrhemstad

cpp/include/cudf/detail/copy_if.cuh

…948_adding_copy_if

rgsl888prabhu added 12 commits October 31, 2019 18:13

changes

86b05ec

Merge branch 'branch-0.11' of https://github.com/rapidsai/cudf into 2…

470d7a2

…948_adding_copy_if

primary changes

0c6ad10

files are compiling

8331791

changes for test cases

f46ae31

merge 0.11

d055987

changes to filter

216b367

changes to return column

26b1b32

ahh working

8e755fb

All set

b743175

documentation

7cfccc1

code changes and test cases

a0d0e61

rgsl888prabhu requested review from a team as code owners November 5, 2019 21:05

CHANGELOG

a72ebe7

rgsl888prabhu self-assigned this Nov 5, 2019

rgsl888prabhu added libcudf++ labels Nov 5, 2019

rgsl888prabhu changed the title ~~[WIP] Define and implement new stream_compaction APIs copy_if and drop_null~~ [REVIEW] Define and implement new stream_compaction APIs copy_if and drop_null Nov 5, 2019

rgsl888prabhu requested review from harrism and jrhemstad November 5, 2019 22:51

harrism requested changes Nov 6, 2019

View reviewed changes

rgsl888prabhu added 4 commits November 7, 2019 10:24

adding apply_boolean_mask

ca97b2e

Adding apply_boolean_mask with test case

8a1cea6

documentation

8dd75d8

review changes

664afea

removing the factory method

703221a

trevorsm7 reviewed Nov 18, 2019

View reviewed changes

cpp/include/cudf/detail/gather.cuh Outdated Show resolved Hide resolved

trevorsm7 reviewed Nov 18, 2019

View reviewed changes

cpp/include/cudf/detail/gather.cuh Show resolved Hide resolved

review changes

856794d

harrism requested changes Nov 19, 2019

View reviewed changes

review changes

df0d9aa

jrhemstad requested changes Nov 19, 2019

View reviewed changes

review changes

8a77cfb

harrism and others added 2 commits November 20, 2019 12:18

Merge branch 'branch-0.11' into 2948_adding_copy_if

a1e1259

string test for drop_duplicates

f5687b2

harrism requested a review from jrhemstad November 20, 2019 20:03

harrism reviewed Nov 20, 2019

View reviewed changes

cpp/tests/stream_compaction/drop_duplicates_tests.cu Show resolved Hide resolved

harrism approved these changes Nov 20, 2019

View reviewed changes

rgsl888prabhu removed the 5 - Merge After Dependencies label Nov 21, 2019

adding string test for cudf::gather

4129ed5

jrhemstad requested changes Nov 21, 2019

View reviewed changes

cpp/include/cudf/detail/copy_if.cuh Outdated Show resolved Hide resolved

cpp/include/cudf/detail/copy_if.cuh Outdated Show resolved Hide resolved

review changes

91ef036

jrhemstad requested changes Nov 21, 2019

View reviewed changes

cpp/include/cudf/detail/copy_if.cuh Outdated Show resolved Hide resolved

rgsl888prabhu added 2 commits November 21, 2019 12:09

Merge branch 'branch-0.11' of https://github.com/rapidsai/cudf into 2…

ae85f57

…948_adding_copy_if

review changes

5e64ad3

jrhemstad approved these changes Nov 21, 2019

View reviewed changes

rgsl888prabhu added 5 - Ready to Merge Testing and reviews complete, ready to merge and removed 3 - Ready for Review Ready for review by team labels Nov 21, 2019

harrism merged commit bcb7bf1 into rapidsai:branch-0.11 Nov 21, 2019

galipremsagar mentioned this pull request May 8, 2020

[FEA] Add a new parameter for all_integer & all_float to evaluate null values as True #5136

Closed

wence- mentioned this pull request Nov 27, 2023

Implement IndexedFrame.duplicated with distinct_indices + scatter #14493

Merged

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[REVIEW] Define and implement new stream compaction APIs `copy_if`, `drop_nulls`, `apply_boolean_mask`, `drop_duplicate` and `unique_count`. #3303

[REVIEW] Define and implement new stream compaction APIs `copy_if`, `drop_nulls`, `apply_boolean_mask`, `drop_duplicate` and `unique_count`. #3303

rgsl888prabhu commented Nov 5, 2019 •

edited

Loading

codecov bot commented Nov 5, 2019 •

edited

Loading

harrism left a comment

rgsl888prabhu commented Nov 6, 2019

harrism commented Nov 6, 2019

harrism Nov 18, 2019

harrism commented Nov 20, 2019

harrism left a comment

rgsl888prabhu commented Nov 21, 2019

[REVIEW] Define and implement new stream compaction APIs copy_if, drop_nulls, apply_boolean_mask, drop_duplicate and unique_count. #3303

[REVIEW] Define and implement new stream compaction APIs copy_if, drop_nulls, apply_boolean_mask, drop_duplicate and unique_count. #3303

Conversation

rgsl888prabhu commented Nov 5, 2019 • edited Loading

codecov bot commented Nov 5, 2019 • edited Loading

Codecov Report

harrism left a comment

Choose a reason for hiding this comment

rgsl888prabhu commented Nov 6, 2019

harrism commented Nov 6, 2019

harrism Nov 18, 2019

Choose a reason for hiding this comment

harrism commented Nov 20, 2019

harrism left a comment

Choose a reason for hiding this comment

rgsl888prabhu commented Nov 21, 2019

[REVIEW] Define and implement new stream compaction APIs `copy_if`, `drop_nulls`, `apply_boolean_mask`, `drop_duplicate` and `unique_count`. #3303

[REVIEW] Define and implement new stream compaction APIs `copy_if`, `drop_nulls`, `apply_boolean_mask`, `drop_duplicate` and `unique_count`. #3303

rgsl888prabhu commented Nov 5, 2019 •

edited

Loading

codecov bot commented Nov 5, 2019 •

edited

Loading